TAP Gibbs Free Energy, Belief Propagation and Sparsity
نویسندگان
چکیده
The adaptive TAP Gibbs free energy for a general densely connected probabilistic model with quadratic interactions and arbritary single site constraints is derived. We show how a specific sequential minimization of the free energy leads to a generalization of Minka’s expectation propagation. Lastly, we derive a sparse representation version of the sequential algorithm. The usefulness of the approach is demonstrated on classification and density estimation with Gaussian processes and on an independent component analysis problem.
منابع مشابه
Belief Optimization for Binary Networks: A Stable Alternative to Loopy Belief Propagation
We present a novel inference algorithm for arbitrary, binary, undirected graphs. Unlike loopy belief propagation, which iterates fixed point equations, we directly descend on the Bethe free energy. The algorithm consists of two phases, first we update the pairwise probabilities, given the marginal probabili ties at each unit, using an analytic expres sion. Next, we update the marginal proba ...
متن کاملPerturbative Interpretation of Adaptive Thouless-Anderson-Palmer Free Energy
In conventional well-known derivation methods for the adaptive Thouless-Anderson-Palmer (TAP) free energy, special assumptions that are difficult to mathematically justify except in some mean-field models, must be made. Here, we present a new adaptive TAP free energy derivation method. Using this derivation technique, without any special assumptions, the adaptive TAP free energy can be simply o...
متن کاملMulti-Component-Multiphase Flash Calculations for Systems Containing Gas Hydrates by Direct Minimization of Gibbs Free Energy
The Michelsen stability and multiphase flash calculation by direct minimization of Gibbs free energy of the system at constant temperature and pressure, was used for systems containing gas hydrates. The solid hydrate phase was treated as a solid solution. The fugacities of all components of the hydrate phase were calculated as a function of compositions by the rearranged model of van der Wa...
متن کاملThe DLR Hierarchy of Approximate Inference
We propose a hierarchy for approximate inference based on the Dobrushin, Lanford, Ruelle (DLR) equations. This hierarchy includes existing algorithms, such as belief propagation, and also motivates novel algorithms such as factorized neighbors (FN) algorithms and variants of mean field (MF) algorithms. In particular, we show that extrema of the Bethe free energy correspond to approximate soluti...
متن کاملStable Fixed Points of Loopy Belief Propagation Are Minima of the Bethe Free Energy
We extend recent work on the connection between loopy belief propagation and the Bethe free energy. Constrained minimization of the Bethe free energy can be turned into an unconstrained saddle-point problem. Both converging double-loop algorithms and standard loopy belief propagation can be interpreted as attempts to solve this saddle-point problem. Stability analysis then leads us to conclude ...
متن کامل